fix greatest/least#25715
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
XuPeng-SH
left a comment
There was a problem hiding this comment.
Request changes: the mixed-temporal resolver is not permutation-invariant and can return a wrong value.
[P1] Preserve fractional-second precision when resolving mixed temporal Oids
leastGreatestTemporalItemType replaces the selected type only when rank > bestRank, so two DATETIME (or TIMESTAMP) inputs with different scales keep whichever one appears first. In addition, the all-temporal branch returns a default DATETIME(0). This is not only a metadata issue: varlen peers are parsed and formatted with temporalItemType.Scale, so an earlier low-scale operand rounds a later string before comparison and can change the winner.
A focused reproduction at this head:
DATETIME(1) '2020-01-01 00:00:00.1',
DATETIME(2) '2020-01-01 00:00:00.24',
VARCHAR '2020-01-01 00:00:00.25'
GREATEST returns 2020-01-01 00:00:00.3, while the expected value is 2020-01-01 00:00:00.25. Reordering equal-rank temporal operands changes the behavior. Separately, DATE + DATETIME(1) + DATETIME(6) resolves to DATETIME(0) and selects a scale-1 temporal item instead of preserving scale 6.
Please derive temporal metadata across the complete temporal input set (including max FSP for the chosen packed/common type) rather than keeping the first equal-rank type, and carry that scale into the all-temporal result type. Add permutation tests for both LEAST/GREATEST covering mixed Oids, same-rank different scales, string/JSON peers, binder return metadata, and executor values.
I also audited null/error cleanup, select-list paths, bounded allocations, and wait/ownership behavior; I did not find a separate leak, hang, or unbounded-growth blocker. The focused tests already in the PR pass, but they cover same-Oid scale alignment and therefore miss this mixed-Oid path.
XuPeng-SH
left a comment
There was a problem hiding this comment.
Requesting changes: one blocking performance issue remains.
[P1] Hoist temporal vector accessors out of the row loop (pkg/sql/plan/function/func_builtin_leastgreatest.go:1060-1081). leastGreatestDatetimeValue is called from the inner row × argument loop, but every fixed temporal case calls vector.GenerateFunctionFixedTypeParameter again. Those helpers return wrapper objects, so this turns a column scan into allocation work per cell. On this exact head, an independent testing.AllocsPerRun probe with 8,192 rows and two fixed temporal arguments measured 16,385 allocations per execution (approximately one allocation per argument per row). A million-row two-argument GREATEST/LEAST therefore creates roughly two million short-lived objects, causing avoidable GC and memory pressure.
Please build/reuse the typed readers once per parameter before iterating rows (or use another batch-scoped accessor representation), keep the per-row fixed-temporal path allocation-free, and add a benchmark/allocation regression showing allocations scale with the argument count per batch rather than rows × arguments.
The previous mixed-temporal FSP/permutation issue is fixed: resolution now scans the complete temporal set for max scale, and the binder, constant-fold, executor, NULL/select-list, and permutation tests pass. I also re-ran the full pkg/sql/plan/function suite, focused pkg/sql/plan tests, and focused race tests successfully. The Q1-Q3 unhappy-path audit found no separate retained-resource leak, blocking/wait cycle, or unbounded retained-growth issue.
…ixone into 0713-fix-greatest
已经修改 |
XuPeng-SH
left a comment
There was a problem hiding this comment.
Approved on exact head 2ff064429de94a5c84dee712a1fbdd7292ddb060.
The two previously blocking issues are closed:
- Mixed-temporal resolution now derives maximum fractional-second precision from the complete temporal input set, preserving binder/executor metadata and permutation invariance.
- Fixed temporal vector readers are constructed once per argument per batch rather than in the row × argument loop. The added allocation regression covers row scaling; an independent 8,192-row benchmark on this head measured 400 B/op and 4 allocs/op.
I re-audited resolver/executor symmetry, NULL and select-list behavior, numeric/string/binary precedence, JSON/temporal/YEAR dispatch, invalid-type rejection, result metadata, ownership, and bounded-growth behavior. No remaining correctness, leak, hang, or unbounded-allocation blocker was found.
Local verification passed: full pkg/sql/plan/function, focused binder/constant-fold tests in pkg/sql/plan, focused race tests, go vet for both packages, and the temporal benchmark. CI is green.
Non-blocking hygiene: test/distributed/cases/function/builtin.result contains a large unrelated output-format refresh and four trailing-whitespace lines; cleaning that separately would reduce diff noise and future conflicts.
Merge Queue Status
Waiting for
All merge conditions
Waiting for any of
All queue conditions
|
What type of PR is this?
Which issue(s) this PR fixes:
issue #25215
What this PR does / why we need it:
参数类型规则
类型分类
CHAR、VARCHAR、TEXT;ENUM在本函数入口规范为VARCHAR。BINARY、VARBINARY、BLOB。DATE、DATETIME、TIMESTAMP。TIME、YEAR。BIT按 numeric source 处理。'abc'、CAST('abc' AS TEXT)、ENUM_colENUM先转VARCHAR。CAST('abc' AS BINARY)、CAST('abc' AS BLOB)DATE '2020-01-01'、DATETIME '2020-01-01 12:00:00'TIMETIME '10:00:00'YEARCAST(2020 AS YEAR)2、CAST(2.5 AS DOUBLE)、CAST(3 AS BIT)INTERVAL '1' DAY、GEOMETRY_colINTERVAL、GEOMETRY/GEOMETRY32、SET 等不在本次 resolver 支持域。它们参与混合调用时直接失败,不能因为另一个参数是 temporal 或 JSON 而进入特殊 overload。
ENUM是例外:在所有分支前规范为 nonbinary
VARCHAR,复用现有ENUM -> VARCHARcast。已有 executor支持的 UUID、ROWID、
array_float32、array_float64保留同 Oid 直通行为;这些类型的混合组合仍不支持。
决议结果
resolver 对全部非
T_any参数一次性产生以下结果:优先级矩阵
按下表从上到下匹配,第一个命中的规则就是最终规则,不再继续向下匹配。所有“其余
参数”均指去除
T_any后的其他参数。可以先按下面的顺序理解:优先级决策图
读图时最容易错的两点:JSON 在普通 string/binary 规则前被截获;DATE/DATETIME/TIMESTAMP
在
YEAR + numeric前被截获。TIME本身不是 date-bearing temporal,因此GREATEST(TIME '10:00:00', 2)会继续走 text,而不是 packed-date。NULLliteral;全是 NULL 直接返回 NULL。packed-date 子矩阵;它会压过 YEAR + numeric 和普通字符串规则。
最后的 text/binary family。
例如
GREATEST(CAST(2020 AS YEAR), DATE '2020-01-01', 1)在第 5 步命中date-bearing temporal 规则,
1会先转成字符串并按已选的DATE类型解析;它不会进入后面的YEAR + numeric规则。相反,GREATEST(CAST(2020 AS YEAR), 1999)没有 date-bearing temporal,才会命中 year-numeric 规则。
具体约束如下:
json-temporaloverload,保留 JSON 来源。INTERVAL、GEOMETRY等即使全部参数同 Oid 也必须在分派前拒绝,不能落入普通 executor 的
panic("unreached code")。DECIMAL、TIME、DATETIME、TIMESTAMP 的Width/Scale不同时,不属于无 cast 快路径,必须先做 metadata alignment。若 DECIMAL 共同精度超过
DECIMAL256,与普通 numeric resolver 一致,全部转FLOAT64,不能退回第一个DECIMAL type。
packed-date分支保留全部 temporal 参数的原 Oid,并将所有非 temporal参数转换为其矩阵目标字符串类型。这样
JSON + DATE + BIGINT在checkFn、retType和 executor 重建 resolver 时得到相同的参数类型。
ENUM规范为VARCHAR,再识别本函数没有 executor 的 Oid。INTERVAL、GEOMETRY、GEOMETRY32、SET 等直接失败;UUID、ROWID、DATALINK、
array_float32、array_float64仅保留已有 executor 支持的同 Oid 调用。JSON -> BLOB/BINARY/VARBINARYcast;因此 JSON 与 binary stringfamily 的混合调用直接拒绝,不进入 binary compare 域。
YEAR,按 date-bearing temporal mixed处理;
YEAR保留 temporal Oid,numeric/BIT peer 转字符串,不再进入YEAR + numeric/BIT的 numeric resolver。resultType/ 比较域T_anyVARCHAR/ 无GREATEST(NULL, NULL)-> NULLENUMVARCHAR继续决议ENUM参数先 cast 到VARCHARGREATEST(enum_col, 'b');ENUM + DATE继续进入 packed-date 文本解析VARCHAR/ textVARCHAR;普通 overloadGREATEST(JSON_EXTRACT('1', '$'), JSON_EXTRACT('2', '$'))FLOAT64GREATEST(CAST(1 AS BIGINT), CAST(2 AS BIGINT));GREATEST(CAST(1.5 AS DECIMAL(10,1)), CAST(1.49 AS DECIMAL(12,2)))先对齐;GREATEST(DECIMAL(76,75) 0.9, DECIMAL(76,0) 1)转 FLOAT64GREATEST(INTERVAL '1' DAY, INTERVAL '2' DAY);不能因为同为 INTERVAL 而放行CHAR/VARCHAR、numeric、BIT、TIME或YEARVARCHAR/ packed-dateVARCHAR;json-temporal overloadGREATEST(JSON_EXTRACT('"2020-01-02"', '$'), DATE '2020-01-01')TEXT/ textTEXT;普通 overloadGREATEST(JSON_EXTRACT('"b"', '$'), CAST('a' AS TEXT))VARCHAR/ textVARCHAR;普通 overloadGREATEST(JSON_EXTRACT('10', '$'), 2);按文本比较"10"与"2"JSON -> BLOB/BINARY/VARBINARY通用 castGREATEST(JSON_EXTRACT('1', '$'), CAST('1' AS BLOB))GREATEST(JSON_EXTRACT('"2020-01-01"', '$'), DATE '2020-01-01', CAST('x' AS TEXT))GREATEST(CAST(1 AS INT), CAST(2.5 AS DOUBLE))-> DOUBLEDATE/TIME/DATETIME/TIMESTAMP,且至少两种 Oid 混合DATETIME/ packed-datetemporalItemType取最高优先级GREATEST(DATE '2020-01-01', DATETIME '2020-01-01 12:00:00')TEXTTEXT/ 有 date-bearing temporal 则 packed-date,否则 textGREATEST(DATE '2020-01-01', CAST('2020-01-02' AS TEXT))是 packed-date;GREATEST(TIME '10:00:00', CAST('09:00:00' AS TEXT))是 textCHAR/VARCHARVARCHAR/ 有 date-bearing temporal 则 packed-date,否则 textGREATEST(DATE '2020-01-01', '2020-01-02')是 packed-date;GREATEST(TIME '10:00:00', '09:00:00')是 textBLOBBLOB/ 有 date-bearing temporal 则 packed-date,否则 binaryGREATEST(DATE '2020-01-01', CAST('2020-01-02' AS BLOB))BINARY/VARBINARYVARBINARY/ 有 date-bearing temporal 则 packed-date,否则 binaryGREATEST(DATE '2020-01-01', CAST('2020-01-02' AS VARBINARY))VARCHAR/ packed-dateGREATEST(DATE '2020-01-01', 20200102)TIME+ numeric/BITVARCHAR/ textVARCHAR;temporal overloadGREATEST(TIME '10:00:00', 2),比较的是文本"10:00:00"与"2"YEAR+ numeric/BIT,且无 date-bearing temporalGREATEST(CAST(2020 AS YEAR), 1999)YEAR+ date-bearing temporalVARCHAR/ packed-dateGREATEST(CAST(2020 AS YEAR), DATE '2020-01-01')YEAR+TIMEVARCHAR/ textVARCHAR;temporal overloadGREATEST(CAST(2020 AS YEAR), TIME '10:00:00')TEXTTEXT/ textTEXT;普通 overloadGREATEST(CAST('b' AS TEXT), 2)CHAR/VARCHARVARCHAR/ textVARCHAR;普通 overloadGREATEST('10', 2)->'2'BLOBBLOB/ binaryBLOB;普通 overloadGREATEST(CAST('61' AS BLOB), 2),按原始字节比较BINARY/VARBINARYVARBINARY/ binaryVARBINARY;普通 overloadGREATEST(CAST('a' AS BINARY), 2);提升为 VARBINARY,避免隐式补零GREATEST(UUID_col, DATE_col)packed temporal 的
temporalItemType按DATETIME > TIMESTAMP > DATE > TIME > YEAR选择,独立于
resultType。例如DATE + TIME的resultType是DATETIME,但temporalItemType是DATE。数字与 nonbinary string 混合按文本比较,例如
'10'与2比较为'10'与'2'。JSON 规则在“同 Oid 直接成功”之前,因此
JSON + JSON也不会让T_json进入现有leastFn/greatestFn的通用分支。三个容易混淆的优先级对照:
GREATEST(CAST(2020 AS YEAR), 1999)GREATEST(CAST(2020 AS YEAR), DATE '2020-01-01', 1)VARCHARGREATEST(TIME '10:00:00', 2)VARCHARGREATEST(DATE '2020-01-01', 2)2转字符串后尝试按日期解析VARCHARGREATEST(JSON_EXTRACT('1', '$'), CAST('1' AS BLOB))temporal / JSON 处理边界
JSON
JSON -> VARCHARcast,JSON string 会去掉外层引号,其他 JSON 值输出其 JSON 文本表示。需要 JSON 文本输入的矩阵分支复用该转换。这与 MySQL 普通
JSON item 的
val_str()不完全相同:MySQL 源码路径使用 JSON serialized text,JSON string 会保留外层引号。该差异在本方案中作为已知差异保留。
基础设施。
INTERVAL、binary string family 等拒绝规则均由“优先级矩阵”定义。首期不为本函数新增
JSON -> BLOB/BINARY/VARBINARY通用 cast,避免扩大变更面并避免混淆 JSON 文本字节与 MatrixOne 内部 ByteJson 编码。
temporal 混合处理
参考 MySQL 的
Item_func_min_max,非 JSON temporal 混合由resolveLeastGreatestType()选择leastGreatestTemporalFn();JSON 日期混合由leastGreatestJSONTemporalFn()选择同一个 packed-date 执行 helper。两种 overload共享 temporal item 的计算和逐行 packed 比较,但 JSON overload 不从 binder 后的参数
类型重新归并
resultType。运行期逐行比较:非 JSON temporal overload 从 binder 完成局部 cast 后的参数 vector
重建 resolution。
json-temporaloverload 仅从参数 vector 重建 temporal item 和packed-date 比较;其返回类型固定为
VARCHAR。packed-date模式下,DATE、DATETIME、TIMESTAMP、TIME、YEAR都必须能转换为packed datetime。
DATE、DATETIME、TIMESTAMP直接取其 packed 值;TIME使用语句开始时间在会话时区中的日期补齐(与 MySQL
time_to_datetime()一致),不能调用使用即时 UTC 日期的通用
Time.ToDatetime();YEAR转为该年的 1 月 1 日。字符串、JSON文本和 binary/BLOB 字节按
temporalItemType指定的唯一目标类型解析。temporalItemType后,执行器只能调用该类型对应的cast/parse;该转换失败立即返回错误,不能继续尝试其他 temporal 类型。例如最高优先级为
DATE时,'1'必须按 DATE 解析并报错,不能在 DATE 失败后改按 TIME 解析为00:00:01,更不能补成查询当天的 DATETIME。TIME解析仅在已选目标为 TIME 时允许。DATE、DATETIME或TIMESTAMP,按 MySQLcompare_as_dates()的 packed-value 比较模型,对本行每个参数取得 packed datetime。invalid input错误;不模拟 MySQL 的 warning、解析失败值0或原始字符串 fallback。LEAST/GREATEST。固定 temporal resultvector 按
resultType写入;varlen result vector 按temporalItemType格式化输出。TIMESTAMP的转换使用proc.SessionInfo.TimeZone,与现有 timestamp cast 一致;sessiontimezone 缺失时回退
time.Local,不能将 nil location 传给 timestamp 转换。packed-date,按 resolver 返回的 numeric、text 或 binary 模式复用相应executor;
TIME和YEAR不被错误地提升为DATETIME。范围约束
只覆盖
DATE、DATETIME、TIMESTAMP、TIME、YEAR和按VARCHAR处理的ENUM;不覆盖
INTERVAL、SET、GEOMETRY等 MySQL 类型矩阵项。字符集/collation 的完整行为仍保持首期限制。
首期不支持
bytes.Compare,首期只保证 ASCII 文本和原始字节场景。
已知差异
路径会接近 MySQL 行为;JSON 文本比较只保证复用 MatrixOne 当前
JSON -> VARCHAR语义,不保证逐项复刻 MySQL 的 JSON serialized text 语义。差异主要集中在 JSON string、
边缘类型组合、错误处理、charset/collation 和返回 metadata。
BINARY的返回元数据可能保留固定 binary 类型;本方案把混合BINARY + numeric提升为VARBINARY。这是有意的值语义优先:MatrixOne 若 cast到固定
BINARY会补0x00并改变短值。该 Medium 不在本次解决范围。当前不能完全复现。这属于完整 collation 支持,不在本次范围。
JSON -> VARCHAR会去掉外层引号;MySQL普通 JSON item 的
val_str()使用 JSON serialized text,JSON string 会保留外层引号。因此
greatest(json_extract('{"a":"2"}', '$.a'), '10')这类 JSON string 与普通string 混合比较,可能与 MySQL 不一致。首期不为
GREATEST/LEAST新增独立 JSONserialized text cast 路径。
JSON + BLOB/BINARY/VARBINARY当前拒绝。MySQL 可通过field_type_merge进入binary string 域;MatrixOne 当前
JSONcast 不支持 binary family,本次不扩展通用cast。
JSON +日期类 temporal 与TEXT、BLOB、BINARY/VARBINARY的多域组合当前拒绝。MySQL 可继续通过 field-type merge 支持这些组合;本次为避免 JSON-temporal 的返回类型、
text 域和 binary 域同时分叉而明确不覆盖。
YEAR + numeric/BIT通过函数专用 year-numericoverload 将 YEAR 作为四位无符号 numeric source 归并,以保证值范围;返回类型不保证
逐项复刻 MySQL
field_type_merge对 YEAR 的元数据。若同时存在 date-bearing temporal,则按
packed-date规则处理,不走该 numeric 分支。ParseDateCast、ParseDatetime、ParseTime与
ParseTimestamp;其宽松格式、零日期和 warning 行为仍可能与 MySQL 不同。无效日期在本函数中直接返回
invalid input,不实现 MySQL 的 warning/packed-0/字符串 fallback 语义。
bytes.Compare。ASCII 普通字符串基本可对齐,非 ASCII、大小写不敏感 collation 和特殊排序规则可能不同。
LONG_BLOB、固定BINARY、charset/collation 等返回 metadata 不逐项复刻;MatrixOne 使用
BLOB、VARBINARY、VARCHAR、TEXT等近似类型,值语义优先。行为差异评估
首期目标覆盖下列 MySQL 兼容核心路径,预期与 MySQL 行为差异不大:
不追求完全复刻 MySQL 的路径:
因此,如果目标是修复普通 numeric/string 混合、常见 temporal、MatrixOne 现有 JSON
文本转换语义下的比较,方案与 MySQL 的行为差异可控;如果目标是完整复刻 MySQL 全类型
矩阵、JSON serialized text、collation、warning/fallback 和结果 metadata,则还需要更大
范围改造。